Cameras are used to capture images of scenes and in order to make use of the captured images, for example, to navigate a robotic vehicle, to pick up items in a robotic manufacturing system or to estimate distances of objects from a stereo web camera, it is often necessary to have calibration information for the camera being used. For example, this camera calibration information, for a mono-camera, may be a set of five parameters, one for the focal length of the camera, one for the pixel aspect ratio, two for the image coordinates of the principal point, and one parameter describing nonlinear lens distortion.
It is known to calibrate a camera in a laboratory environment by presenting a known target or motion sequence to the camera. However, this is time consuming and expensive. Also, re-calibration is typically necessary because even camera systems which have been constructed to tight tolerances from high-quality components eventually change their characteristics. For example, this might be due to changes in temperature, dropping or jolting the camera, or changes in humidity. In order to re-calibrate the camera an explicit calibration procedure in a laboratory environment can be repeated but this is expensive and time consuming as mentioned above. Also, this type of calibration process is not suitable for systems such as autonomous robots, which must work in unconstrained environments, or for mass-market consumer applications.
Online camera calibration methods have been proposed whereby calibration information is extracted from the images that the system captures during normal operation and is used to continually update system parameters. However, existing online camera calibration methods do not cope well with structure-poor scenes; that is, where the images that the system captures during normal operation comprise little texture and/or 3D structure. Where cameras are used in the home or office environment it is often the case that images captured in normal operation are structure-poor and/or comprise features which hinder online calibration.